Red Hat Enterprise Linux Implementation

Bash Script Creation

Module Topics

  • Bash Shell Scripting Basics

  • Creating Bash Shell Scripts

  • Executing Bash Shell Scripts

  • Displaying Output

  • Using Variables

  • Using Bash Shell Expansion Features

  • Iterating With the for Loop

  • Troubleshooting Shell Script Bugs

Bash Shell Scripting Basics

  • Simple system administration tasks can be accomplished with Linux command-line tools

  • More complex tasks require chaining multiple commands

    • Command line tools can be combined with Bash shell to create powerful shell scripts

  • Bash shell script is executable file composed of list of commands

    • Can be further leveraged by other scripts

  • Proficiency in shell scripting is essential to Linux system administration

    • Especially crucial in enterprise environments

    • Shell scripts can improve efficiency and accuracy of routine tasks

Bash Shell Scripting Basics

Choosing a Programming Language
  • Bash shell scripting not right tool for all scenarios

  • Programming languages have strengths and weaknesses; none is right for every situation

  • Bash scripts are good for tasks that can be accomplished mainly by calling other command line utilities

    • If task involves heavy data processing and manipulation, languages such as Perl or Python are better

  • Bash supports arithmetic operations limited to simple integer arithmetic

    • For more complex arithmetic operations, consider C or C++

  • Bash supports 1D and associative arrays, but Perl and Python have better array functionality

  • Gain experience with shell scripting and programming languages to learn advantages and disadvantages of each

Creating Bash Shell Scripts

  • Create Bash shell script by opening new file in any text editor

  • Advanced editors such as vim or emacs understand Bash shell syntax and provide color-coded highlighting

  • Highlighting is helpful for spotting syntax errors like unpaired quotes and unclosed brackets

Command Interpreter
  • First line of Bash shell script begins with 2-byte notation #!

    • Commonly referred to as a "sharp-bang" or "sha-bang"

    • Technically referred to as "magic pattern"

  • #! indicates that file is executable shell script

  • Path name follows to command interpreter that executes script

  • Bash scripts are interpreted by Bash shell and so begin with:

    #!/bin/bash

Executing Bash Shell Scripts

  • Use chmod and possibly chown to change script file permissions and ownership so script is executable

    • Grant execute permission only to users script is intended for

  • Executable script can be invoked by name on command line

  • If only base name of script file is entered, Bash searches directories in shell PATH variable for first executable file matching name

    • Avoid script names that match other executable files

    • Configure PATH variable correctly so script is first match

  • To display directory script resides in, use which

    [student@server1 ~]$ which hello
    ~/bin/hello
    
    [student@server1 ~]$ echo $PATH
    /usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/student/.local/bin:/home/student/bin

Displaying Output

Using echo
  • Use echo to display arbitrary text by passing text as argument to echo

  • By default, text is directed to STDOUT (standard out)

  • Can be redirected to STDERR (standard error)

    • Example: Use echo to display "Hello, world" to STDOUT

      [student@server1 ~]$ cat hello
      #!/bin/bash
      
      echo "Hello, world"
      
      [student@server1 ~]$ ./hello
      Hello, world
  • Use echo in shell scripts to display information or error messages during script execution

    • Helpful indicator of script progress

    • Can be directed to STDOUT or STDERR or redirected to log file

Displaying Output

  • Good practice to direct errors to STDERR to differentiate error messages from status messages

    [student@server1 ~]$ cat hello
    #!/bin/bash
    
    echo "Hello, world"
    echo "ERROR: Houston, we have a problem." >&2
    
    [student@server1 ~]$ ./hello 2> hello.log
    Hello, world
    
    [student@server1 ~]$ cat hello.log
    ERROR: Houston, we have a problem.
  • echo is useful for script debugging

  • Add echo statements to misbehaving part of script to clarify commands being executed and values of variables invoked

Displaying Output

Quoting Special Characters
  • Some characters and words have special meanings to Bash shell

  • Some situations require literal values rather than the special meanings of these characters

    • Example: # is interpreted by Bash as beginning of comment and is therefore ignored along with everything following on same line

      If "comment" meaning is not desired, then Bash needs to know that # is to be treated as a literal value

  • To disable meaning of special characters and words, use escape character, \, single quotes, '', or double quotes, ""

  • \ removes special meaning of single character immediately following

    [student@server1 ~]$ echo # not a comment
    
    [student@server1 ~]$ echo \# not a comment
    # not a comment

Displaying Output

  • To escape more than one character, use \ multiple times or use ''

    • Single quotes preserve literal meaning of all characters they enclose

      [student@server1 ~]$ echo # not a comment #
      
      [student@server1 ~]$ echo \# not a comment #
      # not a comment
      
      [student@server1 ~]$ echo \# not a comment \#
      # not a comment #
      
      [student@server1 ~]$ echo '# not a comment #'
      # not a comment #

Displaying Output

  • Double quotes are like single quotes but do not preserve literal value of dollar sign ($), back-ticks (`) and backslash (\)

  • The special meaning of \ is retained only when it precedes a dollar sign ($), back-tick (`), double quote ("), backslash (\), or newline

    [student@server1 ~]$ echo '$HOME'
    $HOME
    
    [student@server1 ~]$ echo '`pwd`'
    `pwd`
    
    [student@server1 ~]$ echo '"Hello, world"'
    "Hello, world"
    
    [student@server1 ~]$ echo "$HOME"
    /home/student
    
    [student@server1 ~]$ echo "`pwd`"
    /home/student
    
    [student@server1 ~]$ echo ""Hello, world""
    Hello, world
    
    [student@server1 ~]$ echo "\$HOME"
    $HOME
    
    [student@server1 ~]$ echo "\`pwd\`"
    `pwd`
    
    [student@server1 ~]$ echo "\"Hello, world\""
    "Hello, world"

Using Variables

  • Useful in complex scripts

  • Serve as containers in which shell script stores data

  • Stored data easy to access and modify during script execution

Assigning Values to Variables
  • To assign value to variable, use this syntax:

    VARIABLENAME=value
  • Variable names are typically uppercase letters

    • Can also be numbers, lowercase letters, and underscore character, _

    • Cannot start with number

  • Use = without spaces to assign values

    • Example: Valid variables

      COUNT=40
      first_name=John
      file1=/tmp/abc
      _ID=RH123

Using Variables

  • Integer and string values commonly stored in variables

  • Good practice to quote string values when assigning to variables

    • Space character is interpreted by Bash as word separator when not enclosed in single or double quotes

  • Use of single or double quotes depends on how special characters should be treated

    full_name='John Doe'
    full_name="$FIRST $LAST"
    price='$1'

Using Variables

Expanding Variable Values
  • To recall value of variable, precede variable name with $

    • Process known as variable expansion

  • Value of VARIABLENAME can be referenced with $VARIABLENAME

    • $VARIABLENAME syntax is simplified version of ${VARIABLENAME}

  • There are situations where brace-quoted form must be used

    [student@server1 ~]$ FIRST_=Jane
    
    [student@server1 ~]$ FIRST=John
    
    [student@server1 ~]$ LAST=Doe
    
    [student@server1 ~]$ echo $FIRST_$LAST
    JaneDoe
    
    [student@server1 ~]$ echo ${FIRST}_$LAST
    John_Doe

Using Bash Shell Expansion Features

Command Substitution
  • Replaces invocation of command with output of its execution

    • Allows output to be used in new context such as argument to another command, value of variable, and for loop construct (covered later)

  • Can be invoked by enclosing command in back-ticks: `<COMMAND>`

  • Preferred method is newer $() syntax: $(<COMMAND>)

    [student@server1 ~]$ echo "Current time: `date`"
    Current time is Thu Jun  5 16:24:24 EDT 2014.
    
    [student@server1 ~]$ echo "Current time: $(date)"
    Current time is Thu Jun  5 16:24:30 EDT 2014.

Using Bash Shell Expansion Features

  • Newer syntax preferred because it allows nesting command substitutions

  • Example: Use output of find as arguments for tar and store that output in variable TAROUTPUT

    [root@server1 ~]# TAROUTPUT=$(tar cvf /tmp/incremental_backup.tar $(find /etc -type f -mtime -1))
    
    [root@server1 ~]# echo $TAROUTPUT
    /etc/group /etc/gshadow /etc/shadow- /etc/passwd /etc/shadow /etc/passwd- /etc/tuned/active_profile /etc/rht /etc/group- /etc/gshadow- /etc/resolv.conf

Using Bash Shell Expansion Features

Arithmetic Expansion
  • Use to perform simple integer arithmetic operations

  • Syntax is $[<EXPRESSION>]

  • Arithmetic expressions enclosed within $[] are replaced with results

  • Before evaluation, Bash performs variable expansion and command substitution

  • Nesting of arithmetic substitutions is allowed

    [student@server1 ~]$ echo $[1+1]
    2
    
    [student@server1 ~]$ echo $[2*2]
    4
    
    [student@server1 ~]$ COUNT=1; echo $[$[$COUNT+1]*2]
    4

Using Bash Shell Expansion Features

  • Space characters are allowed

  • Can improve readability in complicated expressions and when variables are used

    [student@server1 ~]$ SEC_PER_MIN=60
    
    [student@server1 ~]$ MIN_PER_HR=60
    
    [student@server1 ~]$ HR_PER_DAY=24
    
    [student@server1 ~]$ SEC_PER_DAY=$[ $SEC_PER_MIN * $MIN_PER_HR * $HR_PER_DAY ]
    
    [student@server1 ~]$ echo "There are $SEC_PER_DAY seconds in a day."
    There are 86400 seconds in a day.

Using Bash Shell Expansion Features

Operator

Meaning

<VARIABLE>++

Variable post-increment

<VARIABLE>--

Variable post-decrement

++<VARIABLE>

Variable pre-increment

--<VARIABLE>

Variable pre-decrement

-

Unary minus

+

Unary plus

**

Exponentiation

*

Multiplication

/

Division

%

Remainder

+

Addition

-

Subtraction

Using Bash Shell Expansion Features

  • Multiple operators evaluated according to precedence

  • To change evaluation order from default, use parentheses to group sub-expressions

    [student@server1 ~]$ echo $[ 1 + 1 * 2]
    3
    
    [student@server1 ~]$ echo $[ (1 +1) * 2 ]
    4
  • Order of precedence from highest to lowest, with equal operators listed together

Operator

Meaning

<VARIABLE>++, <VARIABLE>--

Variable post-increment and post-decrement

++<VARIABLE>, --<VARIABLE>

Variable pre-increment and pre-decrement

-,

Unary minus and plus

**

Exponentiation

*, /, %

Multiplication, division, remainder

+, -

Addition, subtraction

Iterating With the for Loop

  • Repetitive task can take form of action executed multiple times on single target

    • Example: Checking every minute for 10 minutes to see if process has completed

  • Can also take form of action executed single time on multiple targets

    • Example: Backing up each database on system

  • for loop is shell looping construct used for task iterations

    for <VARIABLE> in <LIST>; do
         <COMMAND>
         ...
         <COMMAND> referencing <VARIABLE>
    done
    • Loop processes <LIST> items in order and exits

    • Each list item stored as value of <VARIABLE> while loop executes <COMMAND>

      • Naming of variable is arbitrary

      • <VARIABLE> value is typically referenced by <COMMAND>

Iterating with the for Loop

  • Can enter for loop items directly

  • Can generate for loop items from shell expansions, such as variable substitution

    • Example: Different ways that lists can be provided to for loops

    [student@server1 ~]$ for HOST in host1 host2 host3; do echo $HOST; done
    host1
    host2
    host3
    
    [student@server1 ~]$ for HOST in host{1,2,3}; do echo $HOST; done
    host1
    host2
    host3
    
    [student@server1 ~]$ for HOST in host{1..3}; do echo $HOST; done
    host1
    host2
    host3
    
    [student@server1 ~]$ for FILE in file*; do ls $FILE; done
    filea
    fileb
    filec
    
    [student@server1 ~]$ for FILE in file{a..c}; do ls $FILE; done
    filea
    fileb
    filec
    
    [student@server1 ~]$ for PACKAGE in $(rpm -qa | grep kernel); do echo "$PACKAGE was installed on $(date -d @$(rpm -q --qf "%{INSTALLTIME}\n" $PACKAGE))"; done
    abrt-addon-kerneloops-2.1.11-12.el7.x86_64 was installed on Tue Apr 22 00:09:07 EDT 2014
    kernel-3.10.0-121.el7.x86_64 was installed on Thu Apr 10 15:27:52 EDT 2014
    kernel-tools-3.10.0-121.el7.x86_64 was installed on Thu Apr 10 15:28:01 EDT 2014
    kernel-tools-libs-3.10.0-121.el7.x86_64 was installed on Thu Apr 10 15:26:22 EDT 2014
    
    [student@server1 ~]$ for EVEN in $(seq 2 2 8); do echo "$EVEN"; done; echo "Who do we appreciate?"
    
    2
    4
    6
    8
    Who do we appreciate?

Troubleshooting Shell Script Bugs

  • Typically due to typographical errors, syntactical errors, and poor logic

  • Easiest way to prevent is to catch them during authoring

  • Use text editor with syntactical highlighting

  • Develop good script-authoring practices and adhere to them

Commenting Scripts
  • Use comments at beginning of script to explain script purpose, intended actions, and general logic

  • Use comments throughout script to clarify important or complex sections

    • Comments help when debugging and serve as reminders of script mechanics after time has passed

Troubleshooting Shell Script Bugs

Format to Improve Readability
  • As long as syntax is correct, command interpreter executes commands without regard for structure and formatting

  • Break long commands into multiple lines of smaller code chunks

    • Easier to read and comprehend

  • Align beginning and end of multiline statements to show where control structures begin and end

    • Easier to see if statement is closed properly

  • Indent lines with multiline statements to represent hierarchy of code logic and flow of control structures

  • Use line spacing to separate command blocks to clarify beginning and end of code sections

  • Be consistent with formatting throughout script

Troubleshooting Shell Script Bugs

Before
#!/bin/bash
for PACKAGE in $(rpm -qa | grep kernel); do echo "$PACKAGE was installed on $(date -d @$(rpm -q --qf "%{INSTALLTIME}\n" $PACKAGE))"; done
After
#!/bin/bash
#
# This script queries the RPM database to get information about when
# kernel-related packages were installed on a system.
#

# Variables
PACKAGETYPE=kernel
PACKAGES=$(rpm -qa | grep $PACKAGETYPE)

# Loop through packages
for PACKAGE in $PACKAGES; do
     # Determine package install date and time
     INSTALLEPOCH=$(rpm -q --qf "%{INSTALLTIME}\n" $PACKAGE)

     # RPM reports time in epoch, so need to convert
     # it to date and time format with date command
     INSTALLDATETIME=$(date -d @$INSTALLEPOCH)

     # Print message
     echo "$PACKAGE was installed on $INSTALLDATETIME"
done

Troubleshooting Shell Script Bugs

Assumptions
  • Do not make assumptions about integrity of inputs such as command-line arguments, user input, command substitutions, variable expansions, and file name expansions

    • Use proper quoting and sanity checking

  • Do not make assumptions about actions external to script such as interacting with files and calling external commands

    • Use Bash file and directory tests

    • Perform error checking on exit status of commands

  • Ruling out assumptions can keep script from failing

  • Example: Lines of code that make risky assumptions

    cd $TMPDIR
    rm *
    • If directory change fails, file removal is performed on list of unknown files in unintended directory!

Troubleshooting Shell Script Bugs

Modifying Someone Else’s Script
  • Not everyone agrees on what good practices are

  • It is important to apply guidelines consistently

  • Be aware of individual differences in programming styles and script formatting

  • When modifying another’s script, follow structure, formatting, and practices used by author

    • Imposing your style may make script inconsistent and reduce its readability and maintainability

Troubleshooting Shell Script Bugs

Debug Mode
  • To activate debug mode, add -x to command interpreter in first line of script

    #!/bin/bash -x
  • Another way is to execute script as argument to Bash with -x

    [student@server1 bin]$ bash -x <SCRIPTNAME>
  • Debug mode prints script commands and shell expansions before executing

Troubleshooting Shell Script Bugs

[student@server1 bin]$ cat filesize
#!/bin/bash

DIR=/home/student/tmp

for FILE in $DIR/*; do
    echo "File $FILE is $(stat --printf='%s' $FILE) bytes."
done

[student@server1 bin]$ ./filesize
File /home/student/tmp/filea is 133 bytes.
File /home/student/tmp/fileb is 266 bytes.
File /home/student/tmp/filec is 399 bytes.

[student@server1 bin]$ bash -x ./filesize
+ DIR=/home/student/tmp
+ for FILE in '$DIR/*'
++ stat --printf=%s /home/student/tmp/filea
+ echo 'File /home/student/tmp/filea is 133 bytes.'
File /home/student/tmp/filea is 133 bytes.
+ for FILE in '$DIR/*'
++ stat --printf=%s /home/student/tmp/fileb
+ echo 'File /home/student/tmp/fileb is 266 bytes.'
File /home/student/tmp/fileb is 266 bytes.
+ for FILE in '$DIR/*'
++ stat --printf=%s /home/student/tmp/filec
+ echo 'File /home/student/tmp/filec is 399 bytes.'
File /home/student/tmp/filec is 399 bytes.

Troubleshooting Shell Script Bugs

  • Large output of debug mode may hinder troubleshooting long scripts

  • Debug mode can be enabled for only part of script

  • Useful when source of problem is narrowed down to portion of script

  • To turned debugging on/off at specific points in script, use set -x and set +x

Troubleshooting Shell Script Bugs

  • This example shows previous example script with debugging enabled only for command line enclosed in for loop

    [student@server1 bin]$ cat filesize
    #!/bin/bash
    
    DIR=/home/student/tmp
    
    for FILE in $DIR/*; do
            set -x
            echo "File $FILE is $(stat --printf='%s' $FILE) bytes."
            set +x
    done
    
    [student@server1 bin]$ ./filesize
    ++ stat --printf=%s /home/student/tmp/filea
    + echo 'File /home/student/tmp/filea is 133 bytes.'
    File /home/student/tmp/filea is 133 bytes.
    + set +x
    ++ stat --printf=%s /home/student/tmp/fileb
    + echo 'File /home/student/tmp/fileb is 266 bytes.'
    File /home/student/tmp/fileb is 266 bytes.
    + set +x
    ++ stat --printf=%s /home/student/tmp/filec
    + echo 'File /home/student/tmp/filec is 399 bytes.'
    File /home/student/tmp/filec is 399 bytes.
    + set +x

Troubleshooting Shell Script Bugs

  • To invoke verbose mode, use -v

  • Bash prints each command to STDOUT prior to execution

  • To turn verbose mode on and off at specific points in script, use set -v and set +v

    [student@server1 bin]$ cat filesize
    #!/bin/bash
    
    DIR=/home/student/tmp
    
    for FILE in $DIR/*; do
        echo "File $FILE is $(stat --printf='%s' $FILE) bytes."
    done
    
    [student@server1 bin]$ bash -v ./filesize
    stat --printf='%s' $FILE) bytes."
    stat --printf='%s' $FILE) bytes.
    stat --printf='%s' $FILE
    File /home/student/tmp/filea is 133 bytes.
    stat --printf='%s' $FILE) bytes."
    stat --printf='%s' $FILE) bytes.
    stat --printf='%s' $FILE
    File /home/student/tmp/fileb is 266 bytes.
    stat --printf='%s' $FILE) bytes."
    stat --printf='%s' $FILE) bytes.
    stat --printf='%s' $FILE
    File /home/student/tmp/filec is 399 bytes.
References
  • Man pages: bash(1), magic(5), echo(1), echo(1p), and seq(1)

Summary

  • Bash Shell Scripting Basics

  • Creating Bash Shell Scripts

  • Executing Bash Shell Scripts

  • Displaying Output

  • Using Variables

  • Using Bash Shell Expansion Features

  • Iterating With the for Loop

  • Troubleshooting Shell Script Bugs

Module Completion

Nice job!

Click the button below to complete this module of the course: